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A FURTHER STUDY ON SUBJECTIVE AND OBJECTIVE ' 



INTELLIGENCE FACTORS^ 



Bratfisch, O. A further study on subjective ^.d ob- 
jective intelligence factors. Reports from the'lnsti- 
tute of Applied Psychology, The University of 
Stockholm. 1971. No. 20. - Forty-three students " ' 

participated in a laboratory experiment involving estim- 
ation of qualitative similarity between items- san^oied 
from ten conventional tests of intellectual performance 
Estimated similarity could tentatively'be de'scribed as ' " 
a function of positive hter-test correlationR as determin- 
ed from another group of 123 individuals with the same 
level of education-r-The similarity matrix was treated 
as in multidimensional psychophysics - test items 
being regarded a^s stimuli. Five "subjective" factors were 
found, cprresponding to the "objective" performance > 
Victors extracted from analyses of the correlational data 
The results confirm the findings of a previous study by 
Bratfisch and Ekman. 



, Introduction and probi 



e. m 



A number of works have shown that modern -psychophysical 
?of^ J. described by e.g. Stevens, i960; Ekman and SjOberg, " 
1965; Ekman. 1969) can be very useful tools not only when iealing 
with the ordinary sense modalities but also within other problem - 
areas as, for example, studi-.s on intrasubj.ective relations have 
shown (e.g Eisler. i960; Ekman. Engen. Ktinnapa'a and Lindman. 
1964; Bratfisch, 1969; Ekman and Lundberg, 1970). Other instances 
are a recent study by Magnusson and Ekman (1970) applying psycho - ' 
physAcal methods to the study of personality, and a number of ^n'- 
veEtigations on physical performance (e.g. Borg, 1962) as well as 
on the perceived difficulty of test -items (e.g. Borg and Forsline, 
970) ^'■^^^'^'^^ 5970: Bratfis<fh. Dornic and Bpr^ 
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concerned jit^ ^,, ,f ''""' °^ above-mentioned kind was 

perceived bT-H dimensionality of intellectual performance. as 
f969f ^ thf r P"^°'"'"'« ^^"^^^^^ (Bratfisch and Ekman, 

wide ; 5^f er Int ^1''"^^" '"^^^ '° traditionally 

one hanH ^PP^f.^^^^es - research on intelligence based, on the 

one hand, on correlational investigations, and. on the other hand on 
^lemen^*"''^""' T^^'^^ techniques, Th. subjects (per soL Svfng 
oiaHtiw/l °^ «*=hooling only) were asked to estimate 

qualitative similarity of.test items, i.e. the degree of qualitatWe 
overlap between items sampled from conventional tests^of i^tluectual 
performance. Estimated similarity could be dejBcribed in TfVr.t 
^i^'^-^^'^on of intertest correlation as' d^termi^ed 

Irial&r'S^^^^^^ 

^r^f^^el^^^^^^^^^^^ e^r^rdMSna- 

of the ISo^TmenHnrr °^ l^' P"'""' "'^^^ ' " ^ continuation 

oi tne a^_ve^mentioned work - was to investigate if the relation ore 

IsZ^l artt'ct""" ''"'"'^'^^ ^^^^"^'^ inter -test correlatfo; 
as well as the close correspondence of "subjective" and "objective- 
factors would be true even for subjects with a higher levJof educa ' 
tion A tentative hypothesis was that there might%merge more "sub 
jective" factors than "objective" ones when deiling w"h subjects 

con^cVus .^trat^"-'' °' ^« ^^^^^^^ numb''er of 

task« f« l,v , t "'^^ when .solving different intellectual 

tasks IS likely to increase with increasing level of education as 
people are being trained in these respects. ^^cation. as 



Method 



• *u "^^^ experiinent was carried out in exactly the same way as 
will be^r!r°"'/!"^^ ty^Bratfisch and EJcman. which from no J on 
Xustrfa is tW ^"^^ experiment was performed in 

t^ft . • ^ "° German test battery available referring 
to the system introduced by Thurston^ {1938/l941) the test b-itt/rv 

Ljd " Stockholm; manual. 1971) was translated and at an- 

tltoTLZ %f °^ belonging to the same higl^! ' 

school and to the same grade as the experimental group. The co- 

™^:rfson d;:i"^°" °^ ^« -ec: 



The stimulus situation 



A ■ ^I" above mentioned test battery were 

so a«?J "^'"^^ experiment. The testa wer^ selected 

^ce AlTTth T ^^11-^*'^- factors of intellectual perform- 

hl ch^tli. Il "^"^ conventional in character ahd may. hen^ 

teVs are It^iTa / ""^""V- J" ^^'^ following presentation the 

tests are arranged according to the factors they represent; the ' 
denotation of factors follows, as indicated above, the system intro'- 
duced by Thurstone (1938. 1941). "yaiem iniro 



/ 

y. hence, 
e 



Factor 



(1) 




< 

VerDal comprehension 


(V) 




Oppo sites 


* ^ If ^ 




(3) 


xuciiLiccii. letters 


Perceptual' speed 


(F) 


(4) 


Identical numbers 


^ II ^ 






Multiplications 


Numerical facility 




(6) 


> 

Mixed cofnput at ions - 


^ fi _ 


(7) 


Number series 


Reasoning ability 




(8j 


Matrices 


- " - 




(9) 


Levers 


Spatial ability 


(s) 


(10) 


Surface development 







The tests were adminisf *ired to the subjects under standard 
conditions. The testing session served, among other things, the 
purpose of making the subjects thoroughly famiUar with the tests 
and, thus, providing them with a basis for judging what kind of per- 
formance was required by any particular test. 

In the exp^iment proper, conducted one week later, sample 
Items from the tests were'presented to the subjects, one item from 
each test. The sample items had high and apprcximatelv equal 
solving frequencies, around 88 ^er cent. 

Similarity estimation 

The subjects were instructed to consider each pair of test 
iterns in turn, and to estimate the degree of similarity between the 
kinds of perforrhance represented by these items. They were in- 
structed to base their judgments entirely on qualitative similarity 
between the tasks, disregarding possible perceived differences 
in difficulty of the tests. This instruction is of particular import- 
ance with respect to-the dimensional analysis, which will be present- 
ed later (cf. Ekman, 1963). 

The estimates of similarity were to be given on a percentage 
scale, 100 representing identity and 0 denoting no similarity- at 
all. The subjects were given a number of training trials, with items 
different. from those of the main experiment, in order to ensure a 
correct understanding of the instructions. 

The 45 pairs of test items were presented in two random or- 
ders, so that each subject produced two estimates of each pair and 
thus, altogether 90 estimates. The experiment was conducted in 
a single session lasting 50 minutes on an average. 



Subjects 



Forty-three subjects participated in the experiment. The 
original group consisted of 49 persons, of which six were ex- 
cluded because of extremely variable responses. This was done 
by calculating a coefficient of correlation between the first 45 
estimates and the second 45 estimatee for each subject seDarately 
The coefficients thus obtained were regarded as a measure of the 
^'individual reliability". Subjects with reliability lower than 0.50 
did not qualify. 

All subjects were"8ta dciiU of ^e eh r ^i tr a l high -school in 
Vienna, Austria, in the last form. With but a few exceptions all 
subjects were male. Their age ranged froni 17-26 years, the 
median age being 18 years. 



R e s*u Its 



As in "Study 1", the results have>een analyzed with respect 
to (a) the reliability of the similarity data, (b) the relation between 
correlation and similarity, and (c) the dimensionality of the per- 
ceived qualitative* overlap betweeif items as well as the dimension^ 
ality of ''objective*^ performance. 



Reliability. of similarity data ^ 

In *'Study 1" it was pointed out that a group of e.g. 30«50 
individuals estimating similarity (which with regard to reliability 
could be considered relatively small) corresponds to a much 
^ larger sample used for determining coefficients of correlation. 
This is; among other things, due to the fact that the standard error 
of a central measure, other things being equal, is less than that 
of a coefficient of correlation. Thus the number of subjects in the 
present study, 43, can certainly be considered representative for 
this purpose. 

Two other aspects of the reliability of the similarity data 
are demonstrated in Figs. lA and B. 

In Fig. lA, the medians of the 43 subjects' first 45 simil-^ 
arity estimates are plotted, against the medians of their second 45 
similarity estimates. There is ^o^systematic deviation between the 
two sets^of data from the graph. The scatter is very moderate* The 
coefficient of correlation over the 45 estimates is + 0,95. Though 
we are aware that central measures'^ generally yield higher coeffi- 
cients of Correlation than do raw values we think that the stability - 
of estimation for the group sTs a whole can be regarded as highly ^ 
satisfactory. Since the first and second estimates of similarity do 
not differ systematically, the mean of the two estimates was com- 
puted for each slimulus pair a^d for each subject. The medians of 
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Fig, Illustration of the reliability of the similarity estimates • 
Diagram A shows the medians of second estimates plotted 
against the medians of first estimates* Diagram B shows 
a measure of uncertainty plotted as a function of median 
similarity estimate: the curve represents a parabolic ^ 
function* 

these individual means, shown in Table la, will -be used for all 
further analyses. For purpose of comparison the corresponding 
data from "Study 1", based on 31 subjects, . are given in Table lb. 
The figures 1 to 10 in Tables la and lb refer to the tests denoted 
by these figures in the chapter **method^^ in this report* 



T^le 1 a. 



Similarity estimates 
(present study) 



Table 1 b. 



Similarity estimates 
("Study r) 
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9 ^ 


10 


1 




99 


3S 


30 
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11 
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18 
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10 


1 




100 


68 


75 


20 


25 


18 


20 


10 


16 
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99 




40 


25 
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12 


10 


20 


10 


10 
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100 




75 


60 


16 


23 


2S 


35 


10 


13 
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3S 


40 




97 


16 


21 


15 


28 


15 


12 
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68 


75 




95 
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35 


35 
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30 
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97 
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28 
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16 
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In Fig. 1 B, the mean differences (disregarding signs) be- 
tween the first and the second similarity estimates are plotted 
against the medians of the similarity estimates. The trend of the 
data might tentatively be desc<?ibed by a parabolic function of the 
form 

D = a {s - 8^), ^— {i) 




where denotes the intra -individual variability or uncertainty 

of estimates and s^the degree of similarity (a is a measare.ment 
constant). Similar results have been obtained in other studies 
(e.g. Ekman and Kttnnapas, 1969; Eisler, i960}. 

Similarity and correlation 

Coefficients of intertest correlation for the group of 123 subject 
are shown in Table 2 a. For comparison the 8am_e„^t_a.ic^r about an 
equally -large group of 128 subjects, used in ''Study 1'% are pre- 
sented in Table 2 b. The figures 1 to 10 in Tables 2a and 2b refer 
. again to the tests denoted by' these figures in the chapter *'method" 
in this report^^ 



Tible 2 a.^ Coefficients of Correlation 
(present study) 
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Table 2 b. Coefficients of Correlation 
("Study 1") 
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'0 


1 




78 


40 


33 


^3 


?8 


45 


40 
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27 
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78 
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36 


22 


24 


46 


47 
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27 
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40 


4r 




66 


3? 


32 


44 


51 


-06 


23 
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30 
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^1 
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72 
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36 




74 


27 


18 
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04 
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28 


24 
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39 
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45 
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The coefficients of correlation between the tests in the pres- 
ent study are, compared to those in "Study 1", smaller throughout, - 
as can be iseen from Tables 2a and 2b. Jnspite of this fact the same 
trend wuh regard to the relation between. similarity and correlation 
appears as in "Stu4y V\ 

In Fig. 2 A the medians of the similarity estimates have been 
plotted against coefficients of correlation. S,imilarity is obviously 
growing with correlation, the form of the trend being obscured by 
a considerable scatter. To bring out the trend more clearly, the 
median similarity estimates of Fig. 2 A have been averaged for 
equal successive intervals of the correlation coefficients. The ran- 
ge of coefficients was divided into seven equal intervals, the inter- 
val width being 0. 136. The average data are shown in Fig. 2 B. 



The trend of the data was examined only on the basis^4 points over 
the positive part of the axis representing correlation. 
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Fig# 2, Medians of similarity estimates plotted against corre- 
lation coefficients. Diagram A shows similarity estimates 
averaged for equal successive interva' of the correlation 
coefficients. The curve drawn represents Equation 2. 



The relation between similarity' estimates corresponding to 
positive coefficients of correlation has again^ as in **Study 1", 
been described by the equation 



S = a + 



b r— 



(2) 



where refers to similarity and £to correlation: a and b are 
empirical constants « Obviously, alternative functions could have . 
been fitted to the present data« However, with but 5' points avail- 
able as a computational basis it was considered adequate to apply 
the function found in "Studv 1". ^ . 



Dimensionality of "subjective'^ and "objective** data 

The same^straighforward procedure as in "Study 1" was 
applied for analysing the dimensionality of the two sets, of data, 
i. e, the matrix of similarity estimates was factored (without 
any transformation of the^estimates into cosines) by the method 
of principal, components, the first five factors being rotated to 
simple structure by he Varimax procedure,* 



As a theoretical discussion concerning the use of the "method 
of similarity analysis", developed by Ekman (1954 ), and re- 
vised by the same author (Ekman 1965), .was given in "Study 
f", no further theoretical reasoning in this respect will be 
offered here. 



The correlation matrix was treated in a similar manner. The two 
matrices are shown in Table 3. For comparative reasons the 
corresponding data of *'Study 1" ar^ given in Table 4, 

A far-reachin£{ agreement between all four sets of data can 
be seen by inspection of Tables 3 and 4. This agreement fur- , 
ther illustrated by the sample of factor plots* shown in Fig, 3, 
For each analysis, there are ten plots of which three are chosen 
so that each factor is represented at least once. The two points 
representing the highest loadings of a factor are represented by 
filled circles at which the particular test numbers are indicated. 
The corresponding configurations --of Fig, 3 are rather similar. 

Taole 3. Rotated principal factors (present study) 



Factor obtained by 
Similarity analysis - Correlational analysis 



Test 


V 


P 


N 


R 


S 


h2 


V . 


P 


N 


R 


S 


h2 


^ 1 Synonyms 


,98 


15 


01 


05 


03 


99 


89 


10 


-09 


-01 


02 


80 


2 Opposltes 


98 


15 


^4 


06 


04 


1.00 


77 


-04 


21 


27 


-02 


71 


3 Identical letters 


23 


96 


07 


08 


05 


98 


11 


70 


48 


15 


04 


76 


4 Identrcal nijnbers 


^2 . 


97 


15 


07 


09 


99 


00 


86 


21 


-06 


00 


78 


5 Multiplications 


02 


11 


96 


08 


04 


95 


OS 


22 


86 


-15 


-13 


83 


»6 Nixed computations 


07 


13 


95 


01 


14 


94 


06 


31 


79 


-04 


18 


76 


7 Nunber series 


03 


00 


67 


45 


34 


76 


16 


16 


-03 


76 


08 


64 


8 Matrices 


10 


17 


19 


88 


. 30 


94 


Q4 


-15 


-10 


81 


15 


72 


9 Levers 


03 


09 


15 


08 


92 


88 ' 


-01 


-07 


15 


09 


92 


89 


10 Surface development 


05 


04 


07 


27 


87 


84 


04 


28 


-37 


38 


64 


78 



Table 4. Rotated principal factors {"Study 1") 



Factor obtained by 
Slmflarl^^ana lysis Correlational analysis 

"^est 'V P N ^R S V P N R S 



1 Synonyms 


93 


34 


11 


-04 


07 


1.00 


84 


20 


14 


13 


04 


77 


2 Opposltes 


^6 


26 


07. 


16 


03 


1.00 


83 


22 


09 


20 


03 


78 


3 Identical fetters 


48 


83 


11 


18 


05 


^^6 


23 


76 


15 


17 


07 


69 


4 Identical numbers 


40 


86 


31 


13 


07 


1.00 


17 


74 


25 


17 


-07 


67 


5 Multiplications 


02 


21 


95 


13 


08 


97 


12 


18 


83 


19 


04 


77 


6 Mixed computations 


11 


14 


96 


-02 


18 


98 


11 


19 


83 


00 


-07 


74 


7 Number series^ 


12 


03 


66 


60 


27 


88 


31 


28 


27 


58 


35 


71 


8 Matrices 


13 


21 


09 


87 


33 


^93 


28 


37 


11 


62 


29 


70 


9 Levers 


04 


-01 


16 


16 


95 


96 


-17 


-11 


-08 


18 


71 


58 


10 Surface development 


04 


11 


13 


23 


93 


95 


24 


13 


05 


07 


71 


59 
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ig. 3. Samples of factor plots. Diagram A illustrates the results 
obtained by similarity analysis in the experimental group 
of ^'Study Diagram B shows the corresponding results 
of the present study. Results obtained by factor analy- 
sis of corjr.elation matrices from **Study 1*' and those of 
the present study are represented in Diagrams C and D 
respectively. 



The identification of each of the five factors from tables 
3 and 4 is self-evident in each case. In each analysis the Verbal 
(V), Perceptual (P), Numerical (N), Reasoning (R), and Spatial 
(S) factors emerge, represented by the .same tests. The main 
results from all analyses have been summarized in Table 5. 



Title 5 ldwtfnc»tli» of rouud f«eton by t«tt •fUi hfghHt 'Mdm?5 



Factor idtAtified «s 


Test 




?«ctOr loi<3ings cfctitoed by 








StfltUrlt/ 

(present stwdy) 




Corr^'at^or 
in« lysis 
{present s:u<Jy» 


«f5i}ySlS 
(■Study 


V • Verb! J ctmfirthttmon 


\ S/nonjms 


9S 


93 


8? 


84 




2 Oppo$lt«s 


98 


.96 


77 


35 


? • Ptrceptuil speed 


3 ideotfCAl letters 


.96 


83 


;c 


76 




4 Identful n\0b€r% 


.97 


.86 


86 


74 


N - NunerlCAl «M1fty 


^ HuUiptiutlons 


.96 


9£ 


.fl6 


B3 




6 H\gt<i C(V(IUt«t1onS 


.95 


.96 


79 


.83 


R • Actsonfng AoMfty 


7 twtoer serle* 


.45 ' 


60 


76 . 


58 




8 Matrices 


.88 


.87 


ei 




S - Sp«tU1^ tbiHty 


9 levers 


.92 


.9S 


9? 


?\ 




ID SorfAce 


.87 


93 


64 


7'- 



developnent 



Discussion 



- The findings of the present study show a close correspond- 
ence to the results obtained by Bratfisch and Ekman (1969). 

The tentative hypothesis that more "subjective" factors than 
"objective" ones .should emerge for subjects with a higher- educa- 
tional level did not come true. This is, in a way^ surprising. 
In higher education usually a certain amount of training in different 
ways of solving intellectual tasks is intro4uced^ One is likely to 
expect that this training, certainly conscious to the students, 
should also be reflected in their perception of what kind of abi- 
lity they use when coping with different intellectual tasks. However, 
according to the present results, that is not the case. Similarity 
analysis yielded the same results as conventional factor analysis 
of correlation data. In other words the features of intellectual . 
performance as experienced by the subject himself appear to be 
characterized by the same dimensionality as objective test perform 
ance^ This seems to be true for both people with ordinary, compul- 
sory education and people with a somewhat higher level of educa- 
tion. It .would be interesting to investigate both still higher and 
maybe also lower educational levels in this respect. (We are, hxDw- 
ever, aware of the difficulties the experimenter would probably \ 
encounter with the latter group as the task to estimate qualitativeX 
overlap between items is rather intellectually demanding.) ' \ 



\ 
\ 



• li- 



lt might be argued thsit the subjects, due to their education, 
had some knowledge about psychological intelligence theory and 
that they had given their esttimates thereto. We do not feel that 
this ia likely to have infiuented the subjects'" responses, in any 
ca«c not to a noteworthy extent, as v/e feel that only a thorough 
knowledge of psychological intelligence theory would block or 
influence one% perception in these respects. 

FurtherxDore it might be argued that the subjects were 
estimating the difficulty of the tasks and ^-e^orting similarity 
in accordance with this kind of perc - 3 in *'Study i.**, a 

separate analysis was carried out j ^ j^resent data to prove 
that this was not the case* For each subject and each pair of tests, 
the difference betw^ren the two standard scores on the te%ts was 
calculatedi as was mentioned above, the full tests had been admin- 
istered to the subjects prior to the experiment proper. . If the degree 
of similarity reported by subjects did reflect individual difficulty, 
these data would show a decreasing similarity with increasing 
diffe rence (disregarding signs) in standard scores. Plots of the 
43 X 45 1935 points thus obtained showed a very considerable 
rar\dom ecatter, but there was practically no trend for similarity 
to decrease with increasing performance differences. These data 
are summarised in Table 6. / 



Table 6, Frequency of individual similarity estimates as related 
individual differences between standard test scores 



Standard score Similarity estimates 



di f ference 


05 


15 


25 


35 


46 


55 


65 


•75 


85 


95 
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«1 


47 


33 


29 


20. 


14 


14 


10 


20 


31 


299 
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82 


52 


58 


.56 


35 


25 


26 


31 


42 


574 
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105 


60 


44 


37 


55 


27 


25 


20 


22 


31 


426 


« 3 


71 


56 


34 . 


33 


19 


15 


22 


24 


14 


17 


305 


4 


40 


31 
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12 


11 


16 
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7 


12_^ 
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21 


18 


13 


10 


5 
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11 
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e" 
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6 


12 
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■ 7 
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3 
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.1 
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1 
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Moreover, the similarity estimates were divided \nto tU'O 
classes (0-50 r low; 51-100 = high) as were the differences 
between the standard scx^res (0-3 = low difficulty;' = high 
difficulty). The coefficient of correlation based on this thus ob- 
tained fourfield table was as low as 0.021. Hence, it can be con- 
cluded that judged similarity between the tasks required by the 
different tests is hardly affected by the individual difficulty of 
these tests, and certainly not to any noteworthy extent. In other 
words, the estimates of similarity reflect qualitative attributes 
of the tasks as perceived by the subjects* 

As has been said before the close correspondence of the 
results of the present study to those from "Study 1" is remarkable 
if one considers that the two groups of subjects had different lev-' 
els of education. On the other hand there is no psychological 
^evidence that the two groups were different as far as level of 
intelligence is concerned. The two groups showed by and large the 
^ame kind of performance level onHihe tests used. No difference 
in the "objective" factor structures could be noticed, which could 
have been expected as e.g. the factors Numerical ability and 
Perceptual speed tend to covary to a higher extent at higher educa- 
tional levels and thus often emerge as one factor only. However, 
one cannot deny that subjects with oi^ly 9 years of elementary ed- 
ucation ("Study 1") are likely to be regarded as having a lower 
level of education than do subjects with 13 years of elementary and 
high school education (present study). 

The relation between similarity and correlation has tenta- 
tively been described mathematically, the computations being based 
on positive correlations only. From Fig. 2 B it is quite obvious 
that the two negative points - if included - would have changed the 
particular form of the mathematical function chosen. Something 
seems to be "wrong'** with the estimates corresponding to negative 
correlations. We do feel that the explanation is to be found in the. 
rating technique applied. Similarity estimates can, according to 
the instructions, only vary between 0 (no similarity at all, i.e. 
no covariation att all^i and 100 (identity, i.e. perfect positive cor- 
relation). Thus, che estimates can never indicate a negative over- 
lap betweex. percepts. This problem could probably be solved by 
giving subjects rating scale which ranges from -i to -f 1 as defin- 
ed in correlational terms. However, this is a scaling problem 
worthwhile to be studied more thouroughly. 

The results of the present investigation confirm in all major 
points the results of Bratfisch and Ekmans' study. However, many 
problems, besides those discussed above, remain to be illuminated 
more closely. In particular, future research should aim at in- 
vestigating what effect similarity in layout could have had on the 
estimates of qualitative overlap between the items. This "could be 
done e.g. by giving different intellectual tasks purely verbal for- 
.mulations. Furthermore the relation between qualitative and 
quantitative overlap between items should be examined. Finally, 
it would be of interest to extend the problems studied here in 
other areas, e.g. mof^r skills. 
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